Fisher Information under Restriction of Shannon Information in Multi-terminal Situations*

Fisher information generally decreases by summarizing observed data into encoded messages. The present paper studies the amount of Fisher information included in independently summarized messages from correlated information sources; that is, the amount of Fisher information when sequences x N and yN of N independent observations of random variables x and y are encoded (summarized) independently of each other into messages mx and mr. The problem is to obtain the maximal amount of Fisher information when the size of the summarized data or Shannon message information is limited. The problem is solved in the case of completely compressed symmetric data summarization. An achievable bound is given in the general case. Information geometry, which is a powerful new differential geometrical method applicable to statistics and systems theory, is applied to this problem, proving its usefulness in information theory as well.